Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update aggregation layers documentation #9135

Merged
merged 2 commits into from
Sep 4, 2024
Merged

Conversation

Pessimistress
Copy link
Collaborator

For #9056

Change List

  • Update aggregation layers documentation
  • Add docs for Aggregator classes
  • Add overview page for aggregation-layers module with GPU aggregation remarks
  • What's new & upgrade guide

Copy link
Collaborator

@ibgreen ibgreen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a big improvement.

Longer term wishlist:

  • It would be really nice to emphasize that we have an independently useful aggregator (can used without layers) if that is the case.
  • Also would love to see us use more data table oriented terminology (e.g columns instead of attributes)


This layer creates `AttributeManager` and makes it available for its subclasses. Any aggregation layer can add attributes to the `AttributeManager` and retrieve them using `getAttributes` method. This enables using `AttributeManager`'s features and optimization for using attributes. Also manual iteration of `data` prop can be removed and attributes can be directly set on GPU aggregation models or accessed directly for CPU aggregation.
Returns a string that indicates the type of aggregator that this layer uses, for example `'gpu'`. This method is invoked with each `updateState` lifecycle. If the type string does not match its previous value, the existing aggregator will be disposed and recreated.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be reworded somehow, the words type and method are a little confusing...

Suggested change
Returns a string that indicates the type of aggregator that this layer uses, for example `'gpu'`. This method is invoked with each `updateState` lifecycle. If the type string does not match its previous value, the existing aggregator will be disposed and recreated.
Returns a string that indicates the type of aggregator that this layer uses, for example `'gpu'`. This aggregator is invoked with each `updateState` lifecycle. If the type string does not match its previous value, the existing aggregator will be disposed and recreated.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The phrase "this method" refers to getAggregatorType(), so it doesn't make sense to change it to "this aggregator". This sentence is clearly not explaining the process well, but I don't know how to improve it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The phrase "this method" refers to getAggregatorType(), so it doesn't make sense to change it to "this aggregator". This sentence is clearly not explaining the process well, but I don't know how to improve it.

Sounds good, just thinking:

  • Maybe align the wording getAggregationMethod() then talking about "this method" makes a bit more sense.
  • Maybe the term "lifecycle" here may not be that helpful?
  • Or how about: the aggregation selected by 'getAggregationMethod()' is run during each 'updateState' call.
  • Or: the selected aggregation is run during each 'updateState' call.


During update state, Subclasses of `AggregationLayer` must first call 'super.updateState()', which calls
#### `onAttributeChange` {#onattributechange}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought for future: Could this be a base layer callback?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably require Aggregtor to be defined with a map of dependencies/update triggers. I'll play with some options.


_Aggregation_ is a 2-step process:

1. **Sort**: Group a collection of _data points_ by some property into bins_.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought for future,: maybe we could define two composable interfaces for these two steps?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WebGLAggregator does the two steps in one pass.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WebGLAggregator does the two steps in one pass.

I see. Perhaps one could generalize the code so it could just one of them somehow. For another time...

```ts
aggregator.setProps({
pointCount: 10000,
attributes: {...},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One day we should call this columns. Not sure if we could go that far now....

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the same format as props.data.attributes and we have a bunch of classes/types along that line such as AttributeManager, BinaryAttribute etc. At least for now I don't think we should add more concepts to the mix.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. But I would love to give this an overhaul in deck.gl v10 as part of deep Arrow integration etc.

@Pessimistress Pessimistress force-pushed the x/aggregation-percentile branch 2 times, most recently from cccee98 to d3a5457 Compare September 3, 2024 21:29
Base automatically changed from x/aggregation-percentile to master September 4, 2024 02:37
@coveralls
Copy link

coveralls commented Sep 4, 2024

Coverage Status

coverage: 91.791% (-0.002%) from 91.793%
when pulling 9b6c6ef on x/aggregation-docs
into 8421929 on master.

@Pessimistress Pessimistress merged commit 8a7c3c7 into master Sep 4, 2024
4 checks passed
@Pessimistress Pessimistress deleted the x/aggregation-docs branch September 4, 2024 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants